Field of the Invention
Embodiments of the disclosure presented herein relate to image and video processing and, more specifically, to efficient edge-aware spatio-temporal filtering and optical flow estimation that may be performed in real time.
Description of the Related Art
Edge-aware filters are an important tool in many image and video applications, such as high-dynamic range (HDR) tone mapping, stylization, and detail manipulation. As used herein, filtering generally refers to combining pixel values, over space or time, according to a filter kernel that defines filter weights. Edge-aware filters in particular make use of information about image edges in the filtering process and are space-variant (i.e., their kernel is dependent on the input data).
Traditional spatio-temporal edge-aware filters have two major drawbacks that impede their use in real-time filtering of videos. First, such filters either operate iteratively on complete video volumes (i.e., all the frames in a video) or on sliding windows of video frames, which require the entire video volume or sliding window of video frames to be stored in memory, thereby incurring high processing latency and requiring access to a large memory with high bandwidth. Second, such filters require accurate optical flow information as input in order to align neighboring frames for temporal filtering, but accurate optical flow is difficult to obtain efficiently with traditional techniques.
To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation.